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One-absorbing barrier random walks arising from Luce’s nonlinear 
beta model for learning and a linear commuting-operator model (called the 
alpha model) are considered. Functional equations for various statistics are 
derived from the branching processes defined by the two models. Solutions 
to general functional equations, satisfied by statistics of the alpha and beta 
models, are obtained. The methods presented have application to other 
learning models. 


The two-response, two-event, path-independent, contingent version of 
a number of stochastic models for learning is given by the equations 


QP, with probability p, 
QoD, with probability (1 — p,), 


(1) Pn+1 = 


where Q, and Q, represent transition operators, and p, and (1 — p,) are, 
respectively, the probabilities of responses A, and A, on trial n, A linear 
model discussed by Bush and Mosteller [8] is obtained when the operators 
in (1) are defined by the equations 


(2) Qs? = OD (0 Sas 1), 
QDn = Q2Dn (Oca <1). 


In this paper, this linear model is called the “‘alpha’’ model. A specialization 
of the nonlinear ‘‘beta’’ model proposed by Luce [13] is obtained when the 
operators are defined by the equations: 


ip teem Oe [= $ 4 
3) Op = TTB pp, b= 1% Be > 0; Om eI. 


In terms of the variable v, = p,/(1 — p,) the transition equations for this 
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version of the beta model are 
(4) ee a with probability p, 
n+l — 


Bd, with probability (1 — p,), 
where 


O<v< @; B; > 0; ¢= 1,2. 


In the beta model response probabilities undergo nonlinear rather than 
linear transformations from trial to trial. Since the probabilities of choice 
inevitably enter into the derivation of stochastic properties of the model, 
the methods generally used to derive properties of linear learning models 
do not apply to the beta model. 

Analytical methods applicable to both the alpha and beta models are 
presented in this paper. The approach used is to consider the branching process 
defined by the decision rules of the two models, and from it to formulate 
functional equations for various statistics of interest. Tatsuoka and Mostel- 
ler [15] used a functional equation approach to obtain some statistics for 
the alpha model. Their techniques differ somewhat from those presented 
here; the approach developed here leads to a unified method of attack for 
the alpha and beta models and can be extended to others. 


Some Random Walks Arising from the Beta Model 


In (4), 8: > land 8; < 1 may be identified, respectively, with reward 
and nonreward of the response. If response A, is never rewarded and response 
Ag is always rewarded 8, < 1, 82 < 1. If both responses are always rewarded 
8; > 1, B < 1. If neither response is ever rewarded 6, < 1, 6, > 1. It is 
shown in [11] that these three cases lead to one-absorbing-barrier (OAB), 
two-absorbing-barrier (TAB), and two-reflecting-barrier (TRB) walks. Rig- 
orous proof of the nature of the barriers for these and other random walks 
resulting from the two-alternative, two-outcome beta model is given by 
Lamperti and Suppes [12]. Only the OAB beta model (8, < 1, B < 1) is 
considered in this paper. Except for the case when a; = 1, in the alpha 
model either response diminishes the probability of response A, ; the alpha 
model is a one-absorbing-barrier model. 


Functional Equations for Statistics of the 
One-Absorbing Barrier Models 


The OAB alpha and beta models lead to an asymptotic distribution of 
p, which has all its density at p = 0. (Considering response A, as an error 
on the part of organisms which are learning, this means that all organisms 
eventually learn not to make errors). Additional information about the 
processes is obtained from various statistics. Following the work of Bush 
and Sternberg [9] on a simple single-operator model, the statistics considered 
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are those which describe the rate of approach to the asymptote, such as the 
mean, weighted mean, and variance of the rate of approach; sequential 
statistics concerning runs of responses; other statistics, such as those de- 
scribing the first occurrence of an A, response (success) and the last occur- 
rence of an A, response (failure). Functional equations satisfied by these 
statistics are derived by considering the branching processes shown in Fig. 1. 
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For the analysis which follows, a sequence 2, , %2 , *** , %q of random 
variables is defined such that 


_ }1 if response A, occurs on trial n 
0 if response A, occurs on trial 7. 


The random variables have expectations p, . 


The mean number of A, responses 


In terms of the random variables x, , the total number of A, responses 
in N trials is given by the random variable Xy = > a Z, With expectation 
E(Xx) = ><", p, . In the one-absorbing barrier models, both responses 
decrease the probability of response A; and 


E(X) = lim E(X,) 


is of interest. In fact, by replacing the parameters of the models by 6 = 
max (8; , 62) and a = max (a, , a) finite upper bounds for #(X) in the two 
models are obtained. Now the number Xy of A, responses in N trials starting 
from trial 1 will be equal to the number, Xy-, , of A, responses in (V — 1) 
trials starting from trial 2 if the result of trial 1 is an A, response and be 
equal to 1 + Xy-, if the result of trial 1 is an A, response. Letting ¢ denote 
the expected number of A, responses, the functional equations for ¢ are 
obtained from Fig. 1 to be 


dav, N) = me + oa(Biv, N- 1)] + (1 oF Hees N—- 1) 


ate Rea cr " [oa(Bv, N-1)+1] v7 Ly 7 Pa(Bav eT), 


and 
alP, N) = Pidalarp, i Rie 1) a 1} + ( cia P)balaep, N= 1). 


When N — o these equations become 


(5) dv) = 5 $(Bw) + Ta 6) + ; = =e 
(6) o.(p) = pbalarp) + (1 — p)dalasp) + p. 


Both the above functions must, of course, satisfy the boundary condition 
¢(0) = 
The second moment of the number of A, responses 

Letting @ denote E(Xx) the functional equations for the second moment 
are then, as N > o~, 


(7) sv) = Tarp 206s») ee 5 9e(Bx2) Be grat 5 + 2¢(8)], 
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(8) 6.{p) = p0.Aoxp) =r dl = DP) 9 (asp) + pll + 2,.(oxp)]. 


.6(0) = 0 is a boundary condition. Finite upper bounds exist for 6,(v) and 
é..(p); replacmg the parameters by 8 = max (8, , 8.) and a = max (a; , a) 
the variance of Xy is pay pa(l — p,) which remains finite as N — @ if 
> p, does. Functional equations for higher moments are easily obtained 
in this manner. 

The functional equations for the mean and second moment of the 
number of A, responses have been previously obtained by Tatsuoka [14] 
and Tatsuoka and Mosteller [15]. Their method of derivation is somewhat 
different from that presented here. 


The weighted number of A, responses 


Define the random variable 


N 
Y an. = > (n + W)2n . 
n=l 
Then Yo,y represents the weighted number of A, responses in N trials with 
the weighting function being the trial number 7. From trial 2 on, the weighted 
number of A, responses is >*_, nz, , which by relabeling the random variables 
XZ. ,%3,°'* AS, XZ, -°+- can be represented by the random variable 


N~1 
Yiweay = 2 (n+ Dan = Yow-r + Xy-1. 
n=} 


If ¥ stands for the expectation of the weighted number of A, responses, 
the functional equations are obtained by noting that Yo,v is equal to Y,,y-1 
if the result of the first trial is an A, response and is equal to (1 + Y1,;y~1)) 
if the result of the first trial is an A, response. For an infinite number of trials, 


(®) Yel) = Ty Mal) + PA Vol) + 640), 
(10) valp) = phalaxp) + (1 — p)Waxp) + o.(p)- 


A boundary condition is ¥(0) = 0. 


Number of trials before the first Az response (success) occurs 


Let F, + 1 denote the trial number on which response A, occurs for 
the first time so that F, is the number of trials before the first A, . F; is 
equal to zero if A, occurs on the first trial and is equal to (J -+ F.), where 
F,, denotes the number of trials, before the first A, response occurs, starting 
at trial 2, if trial 1 results in an A, response. Letting v denote the expectation 
of the random variables F, the functional equations for v are 


(11) ve(v) = pilya(B.v) + 1] + [C1 — pO] = Ty 70080) 4 <a . 
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(12) va(p) = pra(arp) + p. 


If p denotes the second moment of the random variables F, the functional 
equations for p are 


(13) pt) = 74 
= 1 ana ri pa(B.v) + 1 cs - {1 + 2ve(B.v)] ; 
(14) Pa(P) = Pralar , p) + p[l + 2v(axp)]. 
Trial number at which last A, response occurs 
Let 


1 if the last A, response occurs on trial n 


0 if no A, response occurs on, or after trial n 
L 
(N + 1) — nif the last A, response occurs on trial N > n. 


Then the random variable LZ, represents the trial number at which the last 
A, response occurs, and by definition LZ, is zero if no A, response occurs 
on any trial. In the following development, the sequence of responses A,A,A, 
denotes the occurrence of A, on the first trial followed by A, on the second 
trial and by A, on the third trial. It is evident that 


L,+1 if A, occurs 

I; +2 if A.A, occurs 
L,=‘5l,+83 if A,A.A, occurs 

I; +4 if A,A,A.A; occurs 

and so on. 


Letting » denote the expectation of the random variables Z, the functional 
equation for yp, is developed from Fig. 1. For an infinite number of trials 


Ba(p) = PlMalary) + 1] + 1 — pjasp[ua(arasp) + 2] 
+ (1 — pil — axp)azp[ua(azp) + 3] + ++: 
= pialarp) + p + (1 — p)asp + (1 — p)(1 — axp)azp + ++: 
+ [(1 — plop + 2(1 — p)(1 — axp)asp 
+ 3(1 — p)(1 — ap)(1 — axp)aip + +++ + (1 — plasppa(marp) 
+ (1 — pil — ap)azpya(uarzp) + -:-]. 


But the term in brackets in the last expression is just (1 — p)ua(aep) as 
may be deduced from the expression for u.(p). Also 


p+(1 — pap + (1 — pil — apy +--+ = 1 Ila — aip). 
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A similar development for ye(v) results in the functional a 


(15) nev) = T+? a . ng(Bv) + 1 . x up(Bav) + 1 ~ ls al a iy)? 


t=0 


(16) wap) = pualoxp) + (1 — p)ualasp) +1 — iat (1 — aip), 


with (0) = 0, since for p = Ono A, response ever occurs. A different deriva- 
tion for the mean of the trial number at which the last A, response occurs in 
the alpha model will be found in Tatsuoka and Mosteller [15]. 

For the expectation of Li it is necessary to consider the expectations of 
(L, + 1)’, etc. Denoting the second moment of the random variables L 
by y the functional equations for y are 


(17) yev) = Tor (8.0) aa aie 3 e(82v) + Eto + [te i = Bp -1| ’ 


and 


(18) ya(p) = pya(ap) + (1 — p)yaloxp) + [ 2a) ay I (1 — axp) — | ’ 


with 7(0) = 0. Functional equations for higher moments of L, can easily 
be generated in the above manner. 


Number of runs, of length j, of A, responses 
The sequence of responses 
A,A,A,°-> A, A, 
j trials 


is termed a run, of A, responses, of length j. Statistics concerning the number 
of runs of A, responses of length exactly equal to j, and of length greater 
than or equal to 7 (7 = 1,2 ---), are of interest. Let R,,; denote the number 
of runs of length 7, which occur between trial n and the termination of the 
process. The total number of runs of length jis then FR, ,; . From the branching 
process of Fig. 1 it is seen that R,,; = Ra, + Ss.;22 , Where 6,,;2 is the 
Kronecker delta function. Letting c; denote the expectation of the number 
of runs of length 7, the functional equation for o;3 is developed from the beta 
model lattice of Fig. 1(a). For an infinite number of trials 


oia(0) = > (11 p.)(d — Prsi[o;9(8i8.0) + 6:.:), 


where 6,,; is the Kronecker delta function. Substituting ¢;9(8,v) for part of 
the expression gives, 


ose) = pioie(Bw) + ( — Pi)o;p(Bv) + I pa — Piss) — Disi(l — pjs2)]. 
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A similar development gives the functional equation for o,;,(p). The 
functional equations are 


(19) oe) = rp ai(Br) + eG oil Ba0) 


i (EME red 
+ I 2 + Bi Ly Biv 1 + 14+ py 
(20) Cial(P) = poja(arp) + (1 — p)oja(asp) 
on a iG . 1) pil rath Quip + 2 ates ”), 
with o,(0) = 0. 


Number of runs of length greater than or equal to j 


Let T,,,; stand for the number of runs of length greater than or equal 
to j of A, responses, which occur from trial n to the termination of the process. 
Then 7, ,; denotes the total number of such runs. Now, for an infinite number 
of trials 


Ti = Ti + bn. 545 C = 2, 3, ve) 


Letting \; be the expectation of the number of runs of length > j, a 
development similar to that previously outlined gives 
7+ 


(21) Asa) = Didse(Bv) + (1 — pi)dsa(B2v) + CL = piss) II Di 


=  vslBo) + ps alba) + 3 Up ges 
(22) Ayal) = Prvelaup) + 1 — B)dvalaup) + (1 — alpen LE pt, 


The expectation of the total number of runs of A; responses in an infinite 
number of trials is obtained when 7 = 1. Denoting this statistic by A, 


(23) dav) = Tay delBv) Tce > hs(Ba0) aa 1 = » “(+ By) = Bw) ’ 


(24) ae = pralaxp) + (1 — p)raloxp) + pl — aup), 
with \(0) = 0. Additional functional equations for other random variables 
of interest, such as runs of A, responses, have been derived in [11]. 

General Functional Equations for the One-Absorbing-Barrier Models 


The functional equations presented for statistics of the beta model 
have the general form 
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v 


1 
(25) fe, Bi , B) = Ls fw, B., B2) + Tp, [ee, B, B2) + gv, B., Bz) 


where 
O<v< om, 0<8, <1, 0<£6<1. 


The term g({v, 6: , 82) is, in general, different for each statistic considered. 
For all except the run statistics 


gv, B , Be) 2 0, 

(26) gO, By ? Bo) = 0, 

lim g@, B, , B.) > 1. 
For these statistics 
(27) f, By , Br) a 0, 

lim IQ, Bi ? Be ene 
Equation (26) does not hold for the run statistics and the boundary conditions 
for the run statistics have to be defined separately. 


The functional equations for statistics of the alpha model are seen 
to have the general form 


(28) -y(p, a , a2) = pylonp, a , 2) + (1 — p)y(arp, a1 , a) + 2(p, a1 , a2) 
where 
O<p<i, O<a <1, O0O<a, <1. 
For the statistics of the alpha model 
2(0, a, , a2) = 0, 
2(1, a, ,a) > 0, 
and the boundary conditions for all the statistics considered are 
(30) y(O, a: , a2) = 0 


and 


(29) 


lim y(p, a, , a2) is finite. 
pl 


The functional equations for the run statistics of the beta model differ 
in nature from the functional equations for the other statistics considered. A 
discussion of the functional equations for the run statistics is presented in [11]. 
The sections which follow present formal solutions to (25) and (28) 
under the boundary conditions (27) and (80) respectively. Theorems con- 
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cerning existence, uniqueness and other properties of the solutions have 
been proved in [11] by methods similar to those of Bellman [3]. Some of these 
theorems are stated here without proof. 


On. the functional equation for the OAB beta model 
Writing f(v, 8, , B2) pon) as f(v), takes the form 


(31) JO) = Tay Mb) + 7 16) + 90), 
where 
ge) 20, 9) =0, lim g@) 2 1, 
and ~ 
f(0) = 0; lim fo) = 


Further, let 0 < 6, < 1;0 < 8 < 1. The cases (8, =1, 8 < 1) and (@, < 1, 
8, = 1) can be considered separately. 
Existence of solution. For any function - define the operator T by 


(32) T-rv) = Da 8) ey > (B20) + gv). 
THEOREM 1. 
fv) = lim T™ g@) 
when the limit exists. 


TueroreM 2. If g(v) ts a monotone increasing function of v, then a solution 
f(v) exists if 


> g(B'v) 


is finite forO <u < @, where 0 < B = max (6, , Be) <1. 

As almost all the g(v) occurring in the beta model first-moment equations 
are monotone increasing functions of » which satisfy the conditions'‘of Theorem 
2, the existence of the mean of most of the random variables introduced for 
the OAB beta model is assured. 

From a proof similar to that for Theorem 2 it follows that when g(v) 
is a monotone increasing function of », 


(33) Y ose) < 10 < YS o0in, 
where 


6B. = max (8: , 6.) and §, = min (A; , 62). 
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Continuity. If | g(v) | is bounded im 0 < » < ©, the solution f(v) is 
continuous. 
Monotonicity. If g(v) is a monotone increasing function of »v, and if 
8, > 8, , then f(v) is a monotone increasing function of v. 
~ Uniqueness. The solution f(v) is unique in0 < v < o. 
On the functional equation for the OAB alpha models 


For the functional equation 


(34) y(p) = p-y(ap) + (1 — p)y(a2p) + 2(p) 


the development of existence, uniqueness and other properties of the solution 
is similar to that for (31). Some properties of y(p) are stated without proof. 
Existence. For any function Q(p), define the operator 


(35) AQ(p) = pQ(arp) + (1 — p)Qlazp) + a(p) 
and let 
lim A” -2(p) |,.1 = c(ar , a). 
THEOREM 3. 
y(p) = lim A™ -2(p). 


Turorem 4. If 2(p) is monotone increasing in p, then 


Dy zaip) < y(p) < x z(oc,p) 
where 
Om = Max (a; , a), @, = min (a, , a). 


Monotonicity. If z(p) is monotone increasing in p, and a, > a, , then 
y(p) is monotone increasing in p. 
Convexity. If 2(p) is convex and_a, > a , then y(p) is convex. 


Solution of the Functional Equation for the OAB Beta Model 


The solution to (31) is obtained by generalizing from solutions of the 
equation for special parameter values. The parameter space of the OAB 
beta model is shown in Fig. 2. One solution for special parameter values is 
derived here. A detailed presentation will be found in [11]. 


TuHroremM 5. Along sides (1) and (2) of Fig. 2, 


= o 


(36) i) = Dy 3 o(eres) T Se. 
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(3) 1,1 
(4) (2) 
Bo 
fe) (1) | 


B J 


Fiaure 2 
Parameter Space of OAB Beta Model 


Proor. Along side (1), 82 = 0, 6: < 1, and only the n = 0 term of the 
summation over 7, is nonzero. The resulting expression is the one obtained 
from the functional equation, for in this case 


10) = 745 160) + 90), 
giving 
KB) = ap oa HOE") + 9(6T0), 


for m = 0, 1, --- , from which the desired result is obtained by successive 
substitution. Along side (2), 8: = 1, 8 < 1, and (86) becomes 


yy g(82) x Tl ‘(1 ee A my) > (1 + Bz) g(62), 


m=0 7=0 
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the last expression being also the one obtained from the functional equation 
for the case 6, = 1, 8. < 1, for which case the functional equation reduces 
to f(v) — f(6v) = (1 + v)9(v). Q.E.D. 

Note that at the point (1, 1) of the parameter space the solutions diverge. 
By letting 6, = 63 , B. = 6 (k = 1,2, ---), solutions along arcs of the form 
(6) and (7) of the parameter space can be obtained. The resulting functional 
equations may be written in the form of q-difference equations for which 
there exists an extensive body of literature [1, 2]. 

Examination of the solutions for various special parameter values sug- 
gests the form of the general solution. The general solution to (31) is given 
by the following theorem. 


THEOREM 6. 
fo) = So Dy Anal) o(6PB). 
where 
Ayo) = 1, 
Ano) = Tt Boh, (m= 1,2, ++), 
Ao = Tay @ = 1,2, ++), 


An. nv) = -> Ao, (0) Ay 0(620) Am-1, n—1(B, 830) (m, n= 1, 2, ns ‘). 


Proor. Substitution in (31) awe 


© 


bs p> An, nv) g(B7B Bv) = = = m .n(Byv) g(Br"*’ Bou) 
mo An n(Bov) (Br, 62" 'v) + g(r) 
so that 


(37) 2 An, ov) g(872) = gv) cae aarvar >> An-1, o(Bw) g(B72), 


which gives 


Ad) = 15 Ans = 75 Aw sol) = THR 


(38) Y Aol) 9(R) = gHey Dy Aon a8) 9B), 
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which gives, 


oe 
Ao) = 1 a , Aoo(B2v) = 1 I+ y? 

= 1 
Ao,»v) = 1 ees v Ao, n=1 Bov) = II él + Biv) ? 


and 


v 


(39) p» > Ann) g(B2 B20) = l + v p> > A n~1,n(B10) g (Bi B20) 


1 © ) 
+ ] + v 2 2. A m,n-1(Bo0) g(B7 B30) . 


The coefficients in this last expression satisfy the difference equation 


v I 
Ann) = 1 + v A -1, nN + y Am n-1(B00) , 
from which follows [11] 


AinQ) = > Ao, 4(0)A1 ,0(B22) Ao. n-4(B1822) 


_ we _ bw 
a Soe ae > Dasaey vy)? 
> Ao, 2(v) A1,0(830) A m—1.-2(81822) « Q.E.D. 


Ann) 


General Solution to the OAB Alpha Model Functional Equation 
Replacing 6, and 8, by a2 and a, in Fig. 2 gives the parameter space of 
the OAB alpha model. The general solution for (34) can be derived [11] in 
a manner similar to that used for the beta model functional equation. The 
solution is given by 


THEOREM 7, 
y(p) = p> yy Dm.n(P) *2(a’iaxp) , 
where 
Bo o(p) = 1, 
bm.o(p) = par ee (m = 1,2, -*+), 
Bo .n(P) a I] (1 = a” "p) (n = 1, 2, Wa ‘), 


Bm lp) = > by, o(@2p) Do ,2(D) Bm—1 nn (1022p) (m,n = 1,2, ---). 
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Proor. The proof is similar to that used for the beta model equation. 
Details are given in [11]. 


Discussion 


Analytical techniques applicable to a class of learning models have 
been presented. Functional equations for various statistics of two learning 
models, viz., Luce’s nonlinear beta model and a linear commuting-operator 
model called the alpha model, have been derived from the branching processes 
defined by the models. 

The results on stochastic properties of Luce’s beta model are new. For 
the alpha model, power series solutions to the functional equations for the 
first and second moments of the total number of A, responses and the trial 
number at which the last A, occurs had been obtained by Tatsuoka and 
Mosteller [15]. However, the techniques of expanding the functions in a 
power series in the variable often fails, as is illustrated by the fact that the 
power series solutions (obtained by Tatsuoka [14]) to the functional equations 
for the first and second moments of the total number of A, responses for 
the one-absorbing-barrier (OAB) beta model are not valid for v > 1. 

By investigating two general equations, the problem of solving the 
individual functional equations for the OAB models was simplified. The 
functional equations for the sequential statistics of the OAB beta model 
do not have the same boundary conditions as the general equation presented 
in this paper, and their solutions require additional investigation. 

Because of the complexity of the expressions obtained for the statistics 
of the OAB models, an attempt was made to find some close bounds which 
could be easily computed. Some upper and lower bounds for statistics of 
the OAB alpha model have been presented in ({11], ch. 5). An upper bound 
for one statistic of the OAB beta model has also been derived in [11], mainly 
to illustrate the methods used to obtain upper bounds for a few statistics 
of the OAB beta model. These methods failed for a number of the statistics. 
Furthermore, a method for the derivation of close lower bounds for the 
OAB beta model remains to be found. 

Empirical tests and comparisons of the beta model with other models 
have been presented by Bush, Galanter, and Luce [6] and Fey [10]. The 
use of statistics such as those derived in this paper for the estimation of 
parameters and for measuring the goodness of fit has been discussed by 
Bush and Mosteller [8], Bush, Galanter, and Luce [6] and by others (see [5]). 
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